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ABSTRACT 


In  this  paper  constant  feedback  control  laws  are  investi- 
gated for  linear  sampled-data  systems.   Derivations  for  the 
minimum-time  problem  and  the  infinite  stage  regular  problem 
which  yield  constant  feedback  optimal  control  laws  are 
presented.   A  case  study  for  the  finite  stage  regulator 
problem  indicates  that  constant  feedback  control,  although 
suboptimal,  may  be  feasible  from  a  design  viewpoint.   Several 
numerical  examples  illustrate  the  results. 


TABLE  OF  CONTENTS 

Page 

LIST  OF  TABLES  5 

LIST  OF  ILLUSTRATIONS  7 

1.  INTRODUCTION  9 

1.1  General  9 

1.2  System  Description  10 

1.3  The  Performance  Criterion  14 

1.4  The  Problem  Statement  15 

2.  TIME  OPTIMAL  OR  DEADBEAT  RESPONSE  16 

2.1  Problem  Definition  16 

2.2  Controllability  17 

2.3  Determination  of  the  Control  Sequence  and       22 
Control  Law 

2.4  Computing  the  Control  Signal  25 

2.5  Arbitrary  Polynomial  inputs  31 

3.  GENERALIZED  PERFORMANCE  INDICES  AND  DYNAMIC  43 
PROGRAMMING 

3.1  General  43 

3.2  Problem  Definition  and  Development  43 

3.3  Recurrence  Relations  51 

3.4  Solution  of  the  Problem  54 

4.  NUMERICAL  EXAMPLES  58 

4.1  Deadbeat  Response  58 

4.2  Infinite  Stage  Regulator  60 

4.3  The  Finite  Regulator  Problem  (CASE  STUDY)        64 


Page 

5.   CONCLUSIONS  73 

5.1   Summary  73 

5.1.1  Deadbeat  Response  73 

5.1.2  The  Infinite  Stage  Regulator  73 

5.1.3  The  Finite  Stage  Regulator  74 

REFERENCES  75 

APPENDIX  I:   Derivation  of  Difference  Equations  76 
Representing  a  Sampled-Data  Dynamic 
System 


LIST  OF  TABLES 

Table  Page 

4-1      System  Trajectory  for  Deadbeat  Example         60 

4-2     Results  of  Case  Study  of  Suboptimal  Control     71 
For  the  Finite  Stage  Regulator  Problem 


LIST  OF  ILLUSTRATIONS 

Figure  Page 

1-1     Block  Diagram  of  the  Plant  12 

2-1     A  Trajectory  in  State  Space  18 

2-2     The  Canonical  and  Orthogonal  Co-ordinate  18 
Systems 

2-3    Vector  Components  27 

2-4    Components  of  the  Canonical  Basis  Vectors  27 
in  the  Orthogonal  Co-ordinate  System 

2-5    Optimal  Regulator  System  30 

2-6    Follow-up  System  Example  35 

2-7    Vector  Block  Diagram  of  Follow-up  35 
System 

2-8    Follow-up  System  Example  with  Auxiliary  37 
Input 

2-9     Follow-up  System  Example  for  Arbitrary  (in  37 
a  class)  Input 

2-10    Signal-flow  Graph  of  a  Plant  in  Canonical  40 
Form 

3-1     Evaluation  of  Performance  Index  44 

3-2     Flow  Graph  for  Polynomial  Example  46 

3-3     Flow  Graph  for  Sinusoidal  Example  46 

3-4    System  Diagram  56 

4-1    Third  Order  System  59 

4-2     Unit  Step  Response  of  Infinite  Stage  62 
Regulator 

4-3     Unit  Ramp  Response  of  Infinite  Stage  63 
Regulator 


Figure  Page 

4-4    Unit  Ramp  Response  of  Infinite  Stage  65 

Regulator  without  Input  Gains 

4-5     Sinusoidal  Response  of  Infinite  Stage  66 

Regulator 

4-6    Sinusoidal  Response  of  Infinite  Stage  67 

Regulator 

i 

4-7     Second-order  System  69 


8 


1 .   INTRODUCTION 
1.1  General 

Optimum  performance  implies  that  a  system  performs  in 
the  "best"  possible  manner  according  to  some  specified  set 
of  criteria.   If  a  criterion  is  established  in  a  mathematical 
form,  called  a  performance  index,  the  "best"  performance  results 
when  this  index  is  minimized  or  maximized.   An  optimum  control- 
ler then  generates  the  sequence  of  control  signals  necessary 
to  obtain  an  extremum  of  the  performance  index. 

This  thesis  investigates  the  optimal  synthesis  of  linear 
sampled-data  control  systems  in  order  to  determine  an  optimum 
or  "suboptimum"  control  law  in  the  form  of  stationary,  linear 
feedback  of  the  plant  variables. 

If  the  performance  index  is  specified  as  a  quadratic 
functional  on  the  plant  variables  and  the  control  variables, 
for  a  linear  system,  the  optimum  control  law  that  is  derived 
to  obtain  an  extremum  of  this  functional  generally  results  in 
a  linear,  time-varying  function  of  the  plant  variables,  i.e., 
multiloop  feedback.   The  physical  realization  of  a  time-vary- 
ing control  law  requires  a  digital  computer,  or  some  other 
time-varying  device.   If  a  stationary  control  law  can  be 
obtained,  the  complexity  of  design,  weight  and  cost  of  the 
controller  is  reduced.   This  is  the  motivation  for  this 
investigation . 


It  will  be  shown  in  two  cases  that  the  optimum  control  law 
is  stationary.   In  a  third  case  the  optimum  control  law  is 
time-varying,  but  a  case  study  shows  that  a  stationary  control 
law  can  be  obtained  that  is  "close"  to  optimum  in  performance. 
1.2  System  Description 

The  physical  system  to  be  controlled  is  called  the  plant, 
and  is  described  by  a  set  of  linear  differential  equations 
with  constant  coefficients.   it  is  controlled  by  certain 
physical  quantities,  called  inputs  or  controls.   The  physical 
variables  of  the  plant  which  can  be  measured  directly  are 
called  outputs. 

By  definition,  the  state  of  a  dynamic  system  is  the 
smallest  collection  of  numbers  which  must  be  specified  at  time 
t=t  ,  along  with  the  control  for  time  t>t  ,  in  order  to  be 
able  to  predict  the  behavior  of  the  system  for  any  time  t=;t0. 
In  explanation,  suppose  the  plant,  which  is  described  by  an 
nth  order  differential  equation,  is  simulated  by  an  analog 
computer.   This  requires  exactly  n  integrators.   The  outputs 
x,,...,xn  of  the  n  integrators,  measured  at  any  instant  of 
time,  are  the  state  of  the  plant  at  that  instant.   The  state 
may  be  regarded  as  a  vector  (nxl  matrix)  and  is  written  as 


x(t)  = 


Kx<t) 


x  (t) 
n  v 


where  the  components  x^ ( I) , . . . , x^ ( t)  are  the  state  variables 
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The  plant  can  be  described  by  the  state  variable  form  of 
the  differential  equation 

x(t)  =  Ax(t)  +  Bu(t)  (1-1) 

and  the  output  equation 

y_(t)  =  Cx(t);  (1-2) 

or,  as  used  in  this  discussion,  by  a  vector  matrix  difference 
equation,  the  discrete  time  solution  of  equation  (1-1) 

x((k+l)T)  =  0(T)x(kT)  +  D(T)u(kT)  (1-la) 

and  the  output  equation 

X(kT)  =  Cx(kT) .  (l-2a) 

Here  u  is  an  m  dimensional  vector  (m<  n)  denoting  the  control 
inputs  to  the  plant;  j£  is  a  p  dimensional  vector  (p<  n) 
denoting  the  outputs  of  the  plant;  A  and  0  are  constant  (nxn) 
matrices;  B  and  D  are  constant  (nxm)  matrices;  C  is  a  constant 
(pxn)  matrix. 

The  methods  required  for  the  evaluation  of  the  transition 
matrix,  (7)  ,  and  the  matrix  D,  from  the  matrices  A  and  B,  and 
the  development  of  equations  (1-la)  and  (l-2a)  from  (1-1)  and 
(1-2),  are  given  in  Appendix  I. 

It  is  assumed  that  the  control  is  exerted  on  the  plant  by 
means  of  a  piecewise  constant  signal  whose  level  can  change 
only  at  a  regularly  spaced  "sampling  points"  or  "sampling 
instants" .   Thus, 
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u*(t)  =  u(kT)  ;   kT^t<(k+l)T 

k  =  0,1,2, .. .  (1-3) 

where  T  is  a  positive  constant  called  the  sampling  period? 
t  =  kT  is  the  kth  sampling  point. 

It  is  customary  to  symbolize  the  sampling  operation  by 
means  of  a  special  element,  the  sample  and  hold  element  (S.H.E.), 
which  converts  the  continuous  signal  u(t)  into  a  sampled 
signal  u*(t)  as  given  by  equation  (1-3).   The  difference  equa- 
tion (1-la)  relates  the  state  x((k+l)T)  at  the  (k+1) st  sampling 
point  to  the  state  x(kT)  at  the  kth  sampling  point  and  to  the 
control  signal,  u_(kT)  .   Fig.  1-1  is  a  block  diagram  representa- 
tion of  equation  (1-la) .   The  blocks,  in  general,  represent 
matrix  multiplication  as  indicated  by  the  double  lined  arrows. 
Where  convenient,  the  sampling  period,  T,  will  be  dropped 
from  the  notation.   It  will  be  understood  that  the  conversion 
from  a  specific  sampling  point  to  an  instant  in  time  will 
require  the  sampling  period.   Equation  (1-la)  and  (l-2a)  will 
now  be  written 

x(k+l)  =  0(T)x(k)  +  D(T)u(k)  (1-lb) 

y(k)  =  Cx(k)  (l-2b) 

It  should  be  noted  that  the  physical  quantities  repre- 
sented by  the  state  variables  and  output  variables  are  contin- 
uous functions  of  time.   The  values  of  the  functions  at  the 
sampling  instant  are  x(k) ,  etc.   For  convenience  and  where  no 
ambiguity  results,  the  notations  x(t) ,  x(k) ,  or  simply  x  may 
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be  used  interchangeably.   This  will  hold  also  for  command 
inputs,  r(t)  and  r  (t)  . 

To  optimally  control  a  plant  governed  by  equation  (1-la) 
it  is  desirable  to  express  the  control  input  u(k)  as  a  function 
of  the  state  and  of  all  present  and  future  desired  states. 
The  concept  of  "desired  state"  is  a  generalization  of  what 
is  usually  called  "command  input"  or  "desired  output".   If 
all  of  the  state  variables  cannot  be  measured  directly,  and 
if  the  plant  is  "observable",  it  is  possible  to  estimate  the 
values  of  unmeasurable  variables  through  use  of  the  Kalman 


Filter 


[l,2 


It  will  be  assumed  that  all  of  the  state 
variables  are  available. 

In  general  terms,  the  design  of  optimum  control  involves 
the  determination  of  a  control  law  or  a  control  sequence  so 
that  the  control  process  can  be  taken  from  a  given  initial 
state,  x(0) ,  to  a  desired  state,  x  ,  in  some  finite  time.   A 
"control  law",  uTx(k),k],  is  the  control  signal  expressed  as 
a  function  of  the  state  variables  of  the  plant  and  possibly 
time„   A  "control  sequence",  u(k)  k=0,l,...,  is  the  control 
signal  expressed  as  a  function  of  discrete  sampling  instants. 
1.3  The  Performance  Criterion 

If  a  system  is  to  be  optimized,  some  criterion  must  be 
chosen  that  determines  how  well  the  system  is  operating.   The 
criterion  established  depends  upon  the  performance  require- 
ments of  the  system.   The  method  used  here  is  to  assign  a 
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cost  to  the  system  at  each  sampling  instant.   The  sum  of 

these  costs  over  the  period  0^  t;f  NT  is  called  the  performance 

index  and  one  form  is  given  by 

JN  =  £  fxT(k)Qx(k)  +  uT(k-l)Ru(k-l)]  (1-4) 

k=lL 

where  N  is  the  number  of  sampling  intervals  to  accomplish  the 

desired  state  transition.   The  matrix  Q  and  the  matrix  R  are 

weighting  factors  which  allow  the  relative  cost  of  the  errors 

and  the  control  effort  to  be  adjusted  to  achieve  the  desired 

performance.   For  optimal  design,  the  performance  index  is  to 

be  minimized.   The  performance  index  will  be  discussed  in 

detail  in  Chapter  3. 

1.4  The  Problem  Statement 

The  problem  can  now  be  stated:   Given  the  system  defined 

by  equation  (1-la) ,  find  the  control  sequence  ja(0) ,u(l) , . . . 

.  .,u_(N-l)  such  that  for  any  initial  state  x(0)  and  some 

desired  output,  a  performance  index  of  the  form  given  by 

equation  (1-4)  is  minimized.   Furthermore,  determine  if  a 

stationary,  linear  feedback  control  law  can  be  established 

such  that  the  total  cost  over  some  period  is  optimum  or  "close" 

to  optimum. 
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2.   TIME  OPTIMAL  OR  DEADBEAT  RESPONSE 
2.1  Problem  Definition 

It  is  desired  that  the  system  achieve  equlibrium  with 
zero  steady-state  error  from  any  initial  state  as  quickly 
as  possible.   in  this  and  following  sections,  in  the  interest 
of  simplicity,  the  desired  state  of  the  plant  is  identically 
zero  for  all  t.   it  will  also  be  assumed  that  the  plant  has  a 
single  input  (m=l) ,  i.e.,  u(k)  is  a  scalar. 

In  order  words,  the  object  of  control  is  to  transfer  any 
initial  state  x(0)  to  the  equilibrium  state  x(N)=0  in  a 
minimum  number  of  sampling  periods.   It  is  desired  to  find  the 
control  sequence,  u(0),  u (1) , „ „ . , u (N-l) ,  and  the  control  law 
which  will  accomplish  this.   A  derivation  of  the  control 
sequence  is  given  from  which  a  geometric  interpretation  can 
be  obtained.   The  result  is  a  stationary  control  law. 

This  problem  may  also  be  stated  in  terms  of  the  performance 
index 

JN  =  xT(N)Q£(N)  (2-1) 

where  Q  is  the  identity  matrix  and  N  indicates  the  final 
stage  of  the  process.   The  problem  then  is  to  find  the  control 
sequence  such  that  N  is  the  smallest  integer  for  which  ^=0 
and  x(t)=£  f°r  aH  t*NT.   Chapter  3  will  show  a  dynamic 
programming  solution  of  this  problem. 
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2.2  Controllability 

A  plant  is  said  to  be  completely  controllable  if  there 
exists  a  control  signal  u(t),  defined  over  a  finite  interval 
t  <  t<  t  ,  which  moves  the  process  from  initial  state  x^o^ 
to  a  desired  equilibrium  state  x(t,  )  in  time  interval  t-i-t  . 
In  the  present  case  the  state  x^l)  is  the  state  0_. 

A  necessary  and  sufficient  condition  for  complete  control- 
lability in  the  discrete-time  case  may  be  stated  as  follows: 
An  nth  order  linear  discrete-time  process  with  a  scalar 
control  is  completely  controllable  if,  and  only  if,  the 
vectors 

sx  =  0  (-T)d(T) 
s±    =  0  (-2T)d(T) 

sn  =0  (-nT)d(T)  (2-2) 

are  linearly  independent  1  .   How  these  vectors  arise  will  be 
shown. 

Consider  first  a  second-order  system  (n=2) .   Any  two 
values  of  the  state  variables  x^ (k)  and  x? (k)  define  a  point 
in  the  x-,  ,  x.-,  plane  which  is  called  the  state  space  of  the 
system.   Analogously,  any  point  in  the  state  space  is  called 
a  state.   The  evolution  of  the  system  from  an  arbitrary  present 
state  (x-^(k),  x^Ck))  can  be  represented  by  a  curve  in  the  state 
space,  called  the  trajectory,  along  which  time  is  a  parameter; 
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x(k) 


x(k  +  l) 


Fig.  2-IA    TRAJECTORY    IN    STATE 
SPACE 


*  £(W 


Fig.  2-2       THE    CANONICAL    AND 
ORTHOGONAL    CO-ORDINATE    SYSTEMS 
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see  Fig.  2-1  where  the  arrow  indicates  the  positive  direction 
of  time.   Since  the  solution  of  a  linear  homogeneous  differ- 
ential equation,  i.e.,  u(t)=0,  is  uniquely  determined  for  all 
finite  values  of  time  by  the  initial  state,  there  is  one,  and 
only  one,  trajectory  passing  through  any  given  point  in  the 
X-,  ,  x2  plane.   The  behavior  of  the  system  may  be  thought  of 
as  the  motion  of  a  point  along  a  trajectory.   If  the  point 
does  not  move  at  all,  the  point  is  said  to  be  an  equilibrium 
state. 

If  v-p  v~  are  the  unit  vectors  along  the  co-ordinate 
axes  in  the  state  space,  then  the  state  vector  at  any  time 
kT  can  be  written  in  the  form  (see  B'ig.  2-2) 

x(k)  =  x-^k);^  +  x2(k)v2  (2-3) 

Now,  from  what  initial  states  x(0)  is  it  possible  to  reach 
the  origin  of  the  state  space  in  one  sampling  interval?   The 
transition  equation,  again,  describing  the  process  is 

x(k+l)  =<f)  (T)x(k)  +  d(T)u(k)  (2-4) 

For  k=0,  equation  (2-4)  yields 

x(D  =(p   (T)x(O)  +(T)u(0)  (2-5) 

Thus,  if  it  is  assumed  that  x(l)=0,  the  initial  state  from 
which  this  equilibrium  condition  can  be  reached  in  one 
sampling  period  is,  solving  equation  (2-5)  for  x(0)  and  using 
equation  (2-2) , 

x(0)  =  -0(-T)d(T)u(O)  =  -u(0)s1  (2-6) 
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(A  property  of  the  transition  matrix  is  that  Uj   (T)  =  (£)(kT)  , 
e.g. ,  (J)'1  (T)  =0(-T)  ?  see  Appendix  I).   Equation  (2-6)  states 
that  if  u(0)  is  not  bounded,  all  initial  states  x(0)  lying 
along  the  vector  s_  can  be  taken  to  x(T)=o  in  one  sampling 
period.   The  states  satisfying  equation  (2-6)  constitute  a 
straight  line  in  the  state  space. 

The  set  of  all  states  from  which  the  origin  can  be 
reached  in  at  most  two  steps  is  that  set  from  which  a  state 
given  by  equation  (2-6)  can  be  reached  in  at  most  one  step. 
For  k=l,  equation  (2-4)  yields 

x(0)  ~  -u(O)0  (-T)d(T)  -  u(l)(p   (-2T)d(T) 
or 

x(0)  =  -u(0)_s   -  u(l)s2  (2-9) 

If  the  vectors  s_-,  and  _s_2  are  linearly  independent,  i.e., 
they  do  not  lie  along  the  same  line  through  the  origin  of  the 
state  space,  they  form  a  basis  in  the  state  space.   That  is, 
any  other  vector  in  the  state  space  can  be  written  as  a 
linear  combination  of  Sj.  and  _s 2  • 

It  can  be  shown  1,5  that  the  vectors 

s-l  =  0(-T)d(T) 

s2  =  0(-2T)d(T)  (2-10) 

are  linearly  independent  if,  and  only  if 

TU  ^  <llX  (2-11) 

where  q  is  any  integer  and  L7  is  the  imaginary  part  of  any  of 
the  complex  poles  of  the  transfer  function  of  the  plant.   This 
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means  that  the  input  must  excite  all  natural  frequencies 
of  the  plant;  in  other  words  no  cancellation  of  poles  must 
be  possible  in  the  transfer  function.   In  illustration 

consider  a  second-order  process  with  the  transfer  function 

1 


G(s)  =  -3- 


+W 


From  a  flow  graph  of  this  process  (see  Chapter  3) ,  or  from 
the  state  equations 

*2  =  "UX1 

and  the  methods  in  Appendix  I,  the  following  matrices  result 


0(T)    = 


cosCJt      sinC^T 
j-sinCjT      cosCl/T 
Using   equation    (2-10) 


d(T)    = 


-cosL/T/(J 
sinCJT/CJ. 


*1  " 


0 


^2  = 


(-costjT  cos2LyT  -  sintjT  sin2(jT)/(j" 
_(-cos(jT  sin2(jT  +  sin(jT  cos2(Jt)  /{J 
By  inspection  it  is  seen  that  if  T(<w/=  q7T'  where  q  is  an 
integer,  s,  =  +.  —o    an<^  hence  the  two  vectors  are  not  linearly 
independent. 

If  the  values  of  the  control  signal,  u(0)  and  u(l),  in 
equation  (2-9)  are  not  bounded,  all  initial  states  x(0) 
lying  in  the  plane  containing  vectors  s_   and  j=   can  be  taken 
to  the  equilibrium  state  x(2)  =  0_  in  two  sampling  periods.   If 
the  values  of  the  control  signal  are  bounded,  it  would  take  a 
longer  time  to  reach  the  equilibrium  state. 
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The  vectors  s,  and  s_  may  be  regarded  as  unit  vectors 
along  the  axes  of  a  new  co-ordinate  system,  x(k)  having  the 
co-ordinates  m  (k)  and  m? (k)  in  this  new  system,  (see  Fig. 
2-2) .   This  co-ordinate  system  will  be  called  the  canonical 
co-ordinate  system. 
2.3  Determination  of  the  Control  Sequence  and  Control  Law 

The  next  step  is  to  determine  the  actual  values  of  thdf 
sequence  u(0),  u(l),  which  will  take  an  arbitrary  initial 
state  x(0)  to  the  origin  of  the  state  space. 

It  was  stated  in  the  previous  section  that  every  initial 
state  can  be  written  in  the  form: 

x(0)  =  m1(0)s1  +  m2(0)s2  (2-12) 

where  the  m. ' s  are  real  numbers.   If  the  process  can  be  taken 
from  the  initial  state  x(0)  to  the  equilibrium  state  in  two 
sampling  periods,  it  can  be  taken  from  the  state  x(]_)  to  the 
equilibrium  state  in  one  sampling  period.   The  state  vector 
x(0)  is  a  two-dimensional  vector  and  the  state  vector  x(l) 
is  a  one-dimensional  vector  in  the  canonical  co-ordinate 
system.   Then^  x(l)    may  be  expressed  as 

x(l)  =  m1(l)s1  (2-13) 

Equation  (2-4)  with  k=0  is 

x(l)  =(/>(T)  x(0)  +  d(T)u(0)  (2-14) 

Substituting  equation  (2-12)  for  x(0) , 

x(l)  =0(T)[m1(O)s1  +  m2(0)s2'+  d(T)u(0)        (2-15) 
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Simplifying  using  equation  (2-10),  the  definitions  of  js   and 
_s  ,  and  properties  of  the  transition  matrix,  it  follows  that 

x(l)  =  m1(O)0(T)(^)(-T)d(T)  +  m2  (0)0  (T)0  (-2T)  d  (T)  +d(T)u(0) 

x(l)  =  m1(0)d(T)  +  m2(0)s^1  +  d(T)u(0) 


or 


x(l)  =  m2(0)s1  +  [m1(0)  +  u(0)]  d(T)  (2-16) 

Hence,  x(l)  will  have  the  form  required  by  equation  (2-13)  if, 
and  only  if, 

u(0)  =  -m-^O)  (2-17) 

and 

m2(0)  =  m1(l)  (2-18) 

If  x(l)  is  to  t>e  reduced  to  zero  in  one  step,  then 

x(2)  =  0  =0(T)x(l)  +  d(T)u(l)  (2-19) 

Substituting  equation  (2-13)  for  x(l) , 

x(2)  =0(T)m1(l)s-  +  d(T)u(l)  (2-20) 

Using  the  definition  s^-,  =0(-T)d(T), 

x(2)  =  [m1(l)  +  u(l)]d(T)  (2-21) 

Therefore,  the  condition  for  x(3.)  to  be  reduced  to  zero  in 
one  step  is 

u(l)  =  -m1(l)  =  -m2(0)  (2-22) 

which  is  the  desired  control  signal  at  t=T.   Hence,  at  every 
sampling  instant,  the  optimal  control  signal  u(k)  is  given  by 

u(k)  =  -m, (k) ,  (k=0,l  for  a  second-order 

process)  (2-23) 
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where  in  general  for  any  state , 

x(k)  =  m1(k)s1  +  m2(k)s2,  k=0,l  (2-24) 

The  effect  of  the  control  will  now  be  determined.  Substi- 
tuting equation  (2-24)  for  x(k)  into  equation  (2-4) ,  and  using 
the  optimal  control  signal  as  given  by  equation  (2-23) , 

x(k+l)  =<^)(T)[m1(k)s1  +  m2(k)s2l-  dfTjm^k) 

Simplifying,  using  equation  (2-10)  ,  the  definitions  of  s_ 
and  s_, 

x(k+l)  =  m1(k)d(T)  +  m2(k)s_1  -  d(T)m1(k) 

x(k+l)  =  m2(k)s1  (2-25) 

Comparing  this  with  equation  (2-24) ,  rewritten  with  k=k+l, 

x(k+l)  =  m  (k+l)^  +  m  (k+l)s_2  (2-26) 

we  see  that  after  the  application  of  the  optimal  control 

m1(k+l)  =  m2(k)  (2-27) 

In  the  general  case  for  an  nth  order  system,  equations  (2-24) 
and  (2-25)  are 

x(k)  =  m^(k)s_1  +  m2(k)j32  +...+  m  (k)s^  (2-24a) 

and 

x(k+l)  =  m2(k)s^1  +  m3(k)s^2  +  ...+  mn(k)s  _1      (2-25a) 
and  therefore 

mi(k+l)  =  mi+1(k)  (2-27a) 

A  geometric  interpretation  of  equations  (2-23  to  2-27a)  is 
as  follows.   The  initial  state  of  the  plant  may  lie  anywhere 
in  the  n-dimensional  state  space.   After  one  sampling  period 
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and  the  application  of  a  control  signal  as  given  by  equation 

(2-23) ,  the  state  can  be  expressed  as  a  linear  combination  of 

only  n-1  linearly  independent  vectors  s,,...,s    ;  in  other 

— -1-     -n-1 

words,  after  one  sampling  period,  the  initial  state  always 
lies  in  an  (n-1) -dimensional  space.   The  initial  state  is 
taken  successively  into  a  space  of  one  less  dimension  during 
each  sampling  period.   Ultimately,  the  state  must  be  in  a 
space  of  dimension  zero  which  is  an  equilibrium  point  in  the 
state  space. 

For  the  second-order  case,  the  initial  state  x(°)  lies 
anywhere  in  the  s_  ,s_  plane.   After  the  application  of  one 
control,  the  state  x(3.)  lies  on  the  line  described  by  the 
vector  _s, .   The  final  control  takes  the  state  to  the  origin. 
2.4  Computing  the  Control  Signal 

According  to  equation  (2-23) ,  the  optimal  control  signal 
is  obtained  by  computing  the  first  component  in  the  canonical 
co-ordinate  system,  m-.  (k)  ,  of  the  state  of  the  plant  at  each 
sampling  instant.   it  will  now  be  shown  that  the  optimal 
control  is  given  by 

u(k)  =  aTx(k)  (2-28) 

where  a.  is  an  nxl  vector  of  constants,  i.e.,  the  control  law 
is  stationary. 

Any  component  of  a  vector  in  one  co-ordinate  system  can 
be  expressed  as  a  linear  combination  of  the  components  of 
the  same  vector  in  a  different  co-ordinate  system.   Thus, 
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it  is  always  possible  to  write,  considering  the  second- 


order  process, 

m1(k)  =  a1x1(k)  +  a2x2 (k)  (2-29) 

where  the  a-  are  real  constants  (see  Fig.  2-3) .   It  is  then 
necessary  to  calculate  these  constants  to  obtain  the  control. 
Observe  that  by  definition, 

x(k)  =  x-^k)^  +  X2^k^9  (2-30) 

and 

x(k)  =  m1(k)s^1  +  m2(k)s_2  (2-31) 

In  matrix  notation,  these  expressions  may  be  equated  and 


written    as 

[^i:  ^2J 

x2(k) 

=    [*l|*2 

"m1(k) 
m2  (k) 

(2-32) 


By  definition 


[±i!*i] 


[0(-T)d(T)  ;0(-2T)d(T)] 
PH   P12' 


LP21   P2  2 


=  H 


(2-33) 


The  basis  vectors  s_^,s_2  are  related  to  the  basis  vectors 


[il',i2]  =  [^l'M 


Pll  p12 
P21  P22 


(2-34) 


This  equation  states  that 

Si  =  Pll-1  +  P21~2 

—2  =  P12-l  +  P22~2    ^see  Fig*  2~4^ 


(2-35) 
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*2 

a2x2(k) 


1\  a,x,(k)         x,(k) 


Fig.  2-3      VECTOR    COMPONENTS 


*2 


22 


21 


Fig.  2-4     COMPONENTS    OF  THE    CANONICAL 
BASIS  VECTORS    IN  THE    ORTHOGONAL 
CO-ORDINATE     SYSTEM 
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Substituting  equation  (2-34)  in  equation  (2-32) , 


V;l:v2 


x1  (k)' 

k2  (k) 


frVQ  ip] 


m1(k) 
m2  (k)_ 


(2-36) 


Note  that  the  vectors  y-,  and  y_2  are  unit  vectors  in  an 
orthogonal  co-ordinate  system,  and  hence,  have  the  values 


Xl  = 

1 

/ 

V 

2  " 

0 

_0_ 

_1 

Therefore  the  matrix 

[^2]  = 

'l 

0" 

w0 

1_ 

is  the  identity  matrix.   Solving  equation  (2-36) , 

-1 


m1(k)" 
Lm2(k}< 


■  M 


x1(k) 

x  (k) 
«-  2V  ■ 


(2-37) 


or  in  general  for  an  nth-order  system, 
m(k)   =   P~1x(k) 


(2-38) 


-1 


where  =  |  s -,  ;  s_   ]  |  _s  1. 


-L* 


If  the  system  is  completely  controllable,  the  required  inverse 
exists.   Thus,  equations  (2-23)  and  (2-38)  together  define  the 


control  law  u 
«-l 


x(k) 


all  a12 


nl 


Letting 


lln 


nn 


.T. 


u(k)  =  -m  (k)  =  -axx(k) 


(2-39) 
(2-40) 


.T 


where  a.   is  the  notation  for  a  vector  transpose  and  is  the 
first  row  of  the  matrix  P~  .   The  optimum  control  law  is 


28 


therefore  constant  feedback  of  the  state  variables  at  sampling 
instants.   Fig.  2-5  shows  the  block  diagram  for  an  arbitrary 
second-order  process  implementing  the  stationary  feedback  law 
when  the  desired  state  is  0_. 

The  control  sequence  can  also  be  found  as  a  function  of 
the  state  x(0)  .   From  equations  (2-23)  and  (2-27a) ,  the 
following  relations  may  be  derived 

u(k)  =  -m^k)  =  -mk+1(0)  (2-41) 

Equation  (2-38)  with  k=0  is 

(2-42) 


m(0)  =  P_1x(0) 


Hence, 


where 


u  =  -P_1x(0) 


(2-43) 


u 


u(0) 

u(l) 


u(n-l) 


(2-44) 

The  optimal  control  was  derived  here  for  a  second-order 
system  but  holds  for  an  nth  order  system  where  the  initial 
state  vector  can  be  expressed  as 

x(0)  =  -11(0)^  -  u(l)s2  _..._  u(n-l)sn  (2-45) 

If  the  n  vectors 

s±   =  0(-iT)d(T),  i  =  1,2, ...,n  (2-46) 

are  linearly  independent  and  if  the  values  u(i),  i=0,l,..., 
n-1,  are  not  bounded,  the  initial  state  x(0)  can  be  brought  to 


29 


X 

oa 

^ 

+ 

(A 

2 
1- 

c/> 

> 

0) 

1 

CM 
X 

i 

— 

o 

to 

X 
O 

o 

l- 
< 

CD 

1 

I 

jliif 

UJ 

S 

: 

3 

CM 

o 
1 

-I 
< 

O 

X 
0) 

1 

1 

(\a 

+ 
IL 

+ 

1 
CM 

iZ 

VlrSiy^ 

■H 

\ 

o 

II 

$ 

_ 

30 


the  equilibrium  state  x(N)c=0  in  at  most  n  samples;  that  is, 
t,:f  nt.   The  control  law  was  shown  to  be  stationary.   For  a 
derivation  with  vector  inputs  see   4 I . 
2.5  Arbitrary  Polynomial  Inputs   5 

The  foregoing  discussion  shows  the  derivation  of  the  time- 
optimal  control  law  when  the  desired  equilibrium  point  with 
zero  error  is  the  origin  of  the  state  space.   The  solution  will 
now  be  extended  for  a  followHip  system.   This  problem  is 
equivalent  to  making  the  first  component,  y  (k) ,  of  the  plant 
output  vector 

y_(k)  =  Cx(k)  (l-2a) 

equal  to  some  desired  output,  r(t)  or  r(k),  in  such  a  way 
that  the  plant  is  in  equilibrium  with  zero  system  error. 

If  r(t)  is  an  arbitrary  function  of  time,  the  problem 
cannot  be  solved  in  general.   However,  when  r(t)  has  the  poly- 
nomial form 

A  P"1 

r(t)  =  g1   +   g2t+  .  .  .  +  g  t    ;  tSO 

=  0  ;  t<  0  (2-47) 

(where  the  g.  are  arbitrary  real  numbers)  the  solution  can  be 

obtained  by  modifying  the  solution  of  the  previous  regulator 

problem. 

Note  that  the  components  of  the  plant  output  vector,  ^(k) , 

are  the  physical  variables  of  the  plant  which  can  be  measured 

directly.   These  components  are  linear  combinations  of  the 
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plant  state  variables.   For  example  y1  may  be  the  sum  of  the 


state  variables  x   and  x~.   It  will  be  assumed  at  this  point, 
for  simplicity,  that 

y-L  =  xx  (2-48) 

It  is  known  that  the  optimal  control  requires  knowledge 


of  all  of  the  plant  state  variables.   It  then  seems  feasible 
to  derive  from  r(t)  a  command  input  state  vector,  r_(t)  ,  to 
which  it  is  desired  to  drive  the  plant  state  vector  so  that 
the  plant  is  in  equilibrium  with  zero  system  error.   Then, 
from  equation  (2-47) ,  form 

rx(t)  =  r(t)  =  gx  +  g2t  +  ...+  g  t? 

r  ft)  =  d£(tl  =  g9  +  2CT  t  +...+  (p-l)g  tP~' 


dt 


•^3 


rp(t)  =  ^FTf^  =  (P_1)  (P-2)...(P-(P-D)gp  (2-49) 

dt 


Also,  note  that 

dPr(t) 

P 


=  0 


Then 


dt 


r(t)  = 


(2-50) 


ri(t) 


rp(t) 

Note  that  equations  (2-49)  require  knowledge  of  the  derivatives 

of  the  input.   This  may  be  difficult  to  obtain  in  some  cases. 

If  the  plant  contains  at  least  p  "free"  integrators  (with- 
out feedback) ,  that  is,  the  plant  has  a  type  number  p,  then 
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the  differential  equations  of  the  plant  in  terms  of  the 
state  variables  are 


xl 

X2 

x2 

= 

x2  , 

XP 

= 

XP+1 

(2-52) 

plus  other  equations  which  do  not  involve  the  state  variables 
x  , ...,xD.   Form  the  modified  state  variables 

1  ir 


xl 

— 

xl 

-  rl 

A 

X2 

x2 

-  r2 

A 

X 

= 

X 

-  r 

P 

p 

P 

A 

X 

i 

— 

X. 

1 

/  P 

p<i<n  (2-53) 

Substituting  equation  (2-53)  into  equation  (2-52)  sho^s  that 

the  modified  state  variables  obey  the  same  differential  equa- 
tions.  Hence,  in  equilibrium  when  x=0, 


xl 

— 

rl 

X2 

= 

r2 

X 

P 

= 

r 
P 

x.   =   0  ,    p<  i<  n  (2-54) 

which  shows  that  the  output  will  be  identical  with  the  input 
after,  at  most,  n  sampling  periods. 
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Thus,  the  regulator  system  which  contains  at  least  p  free 
integrators  is  converted  to  a  follow-up  system  simply  by 
taking  the  difference  between  the  plant  state  x(k)  and  the 
input  state  r_(k)  to  create  the  modified  state  x  prior  to 
multiplying  by  the  feedback  coefficients  a   .   In  matrix 
notation,  then, 

u(k)  =  -aTfx(k)  -  r  (k)]  (2-55) 

(the  components,  r   -i,  .  .  .  ,  r  ,  of  vector  r_  will  be  zero). 

Fig.  2-6  shows  the  block  diagram  for  a  simple  second- 
order  plant  with  one  free  integrator  and,  therefore,  the  input 
r(t)  =  constant  (p=l) .   Fig.  2-7  shows  the  block  diagram  of 
a  follow-up  system  for  the  general  case  where  u(k)  is  given 
by  equation  (2-55)  . 

The  foregoing  solution  states  that  the  plant  must  have 
at  least  p  free  integrators,  that  is,  it  must  be  at  least  of 
type  number  p  in  order  to  follow  an  input  r(t)  that  satisfies 
the  differential  equation 

dP^fc)  =  0  (2-50) 

dtP 

For  example,  a  plant  of  type  number  2  will  follow  only  "step" 
and  "ramp"  inputs  (p^ 2)  with  zero  steady-state  error.   However, 
this  restriction  on  the  allowable  order  of  the  input  to  a  system 
with  a  given  number  of  free  integrators  may  be  relaxed  some- 
what as  follows. 
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Fig.  2- 6     FOLLOW-UP    SYSTEM    EXAMPLE 
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Fig.  2-7      VECTOR    BLOCK     DIAGRAM 
OF    FOLLOW-UP      SYSTEM 
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Consider  the  example  of  Fig.  2-6  but  where  the  command 
input  is  a  "unit  ramp", 

r(t)  =  t 
or,  using  equations  (2-49) 

r(t)  =  rv 

_lj  (2-56) 

The  desired  response  then  is  x-.  (oo)  =  t,  and  x  (oo)  =  1. 
Looking  at  a  signal-flow  graph  representation  of  the  system, 
Fig.  2-8  (disregarding  R) ,  we  note,  however,  that  there  is  a 
feedback  of  x   upon  itself  through  an  integrator  and  gain  of 
-d.   This  means  that  x„  could  not  possibly  hold  the  steady- 
state  value  of  1.   Stated  another  way 

x2  (<x> )    =  u  (oo  )  -d*x2(c>o) 

0  4   0  -  d-  (1)  (2-57) 

We  can  get  around  this  difficulty  by  introducing  an  auxiliary 
input,  R,  as  shown  in  Fig.  2-8.   Equation  (2-57)  could  then  be 
written 

x  (OO)  =  R  =  d-x2(oo) 

0  =  R  -  d- (1)  (2-58) 

Therefore, 

R  =  d- (1)  (2-59) 

It  is  seen,  then,  that  through  the  use  of  an  auxiliary  input 
a  plant  can  be  made  to  follow  a  polynomial  input  of  one  order 
higher  than  the  type  number  of  the  plant.   The  introduction  of 
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Fig.  2-9       FOLLOW-UP      SYSTEM      EXAMPLE 
FOR    ARBITRARY   (IN  A  CLASS)    INPUT 
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the  auxiliary  input  simply  results  in  a  shift  in  the  origin 
of  the  canonical  co-ordinate  system.  The  equilibrium  state 
is  still  reached  in  k<n  stages. 

It  is  seen  from  equation  (2-59)  that,  if  the  auxiliary 
input  is  to  be  implemented  as  shown  in  Fig.  2-8,  the  command 
input  and  its  derivatives  must  be  known  beforehand  and  may  not 
be  arbitrary.   However,  by  implementing  R  as  a  function  of  the 
input  state  vector  r_,    the  system  will  follow  an  arbitrary,  but 
still  restricted,  polynomial  input. 

Continuing  with  the  example  of  Fig.  2-8,  it  is  seen 
that 

u  =  "  all(xl  "  rl)  "  a12^x2  "  r2^  +  R  (2-60) 

where 

R  =  d-r2  (2-61) 

Then  substituting  for  R,  and  rearranging, 

u  =  a,nr-|  +  (a-,  ~  +  d)  r2  -  anxi  ~  ai?x2        (2-62) 

If  the  system  is  redefined  as  shown  in  Fig.  2-9,  and  if 

b,  =  a    and  b„  =  (a,   +  d) ,  it  is  seen  that  u  is  still  given 

by  equation  (2-62)  .   The  input  vector,  r_,  may  now  represent 

an  arbitrary  polynomial,  in  a  class,  and  the  origin  of  the 

canonical  co-ordinate  system  will  be  automatically  shifted 

so  that  the  equilibrium  state  may  be  reached. 

In  general  matrix  notation  the  control,  u(k),  is  given 

by 

u(k)  =  bTr(k)  -  aTx(k)  (2-63) 
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where 

bT  =  lb,  b   . . .b  b   ,  ...b  1  (2-64) 

-     L  1   2     p  p+1     nj 

(the  elements  b  ,n  .  .  .b  ,  where  p<n,  are  zero  and  must  be 
P+-L     n 

included  in  the  vectors  b  and  r_  so  that  they  will  be  conform- 
able with  the  (nxl)  vectors  a  and  x. )   Elaboration  is  required 
for  the  general  formulation  of  the  vector  b. 

It  can  be  shown  |6|  that  for  a  scalar  control,  u(t),  and 
a  completely  controllable  system,  the  coefficient  matrix  A 
of  equation  (1-1)  can  be  written  in  a  canonical  form.   This 
results  in  the  signal-flow  graph  representation  of  any  plant 
as  shown  generally  in  Fig.  2-10  (output  not  shown) .   The 
plant  shown  has  m  free  integrators.   From  the  preceding  example 
it  can  be  seen  that  the  elements  of  the  vector  b  are  formed  as 
follows: 

b.  =  a,  .  +  0  ;  i  =  1,  .  .  .  ,m 
l     li 

b   =  a,_  +  0  ;  p«cm+l 

p     IP       * 

bp  =  alP  +  dm+l  '  P  =  m+1 

b.  =  0  7  J  =  p+l,...,n;  p<n 

where  p  is  the  order  of  the  polynomial  input  and  where  d   . 

is  shown  in  Fig.  2-10.   The  a. .  are  the  optimal  gains  given 
.  li 

by  the  first  row  of  P 

It  has  been  shown  that  a  plant  can  be  made  to  follow  a 
polynomial  input  function  of  order  one  higher  than  the  number 
of  free  integrators  in  the  plant.   This  will  be  illustrated  in 
Chapter  4. 
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The  state  of  the  plant,  x(t)  ,  is  a  continuous  function 
of  time  and  is  being  fed  back  continuously,  while  the  control 
input  is  constant  over  the  sampling  period.   Thus,  if  the 
desired  value  of  any  of  the  state  variables  is  not  a  constant, 
and  if  it  is  being  fed  back  to  the  input,  then  the  error  at 
the  input  will  not  be  zero  at  all  times  and  therefore  the 
system  will  not  be  in  equilibrium.   This  will  also  be  illustrated 
in  Chapter  4. 

If  the  plant  output  is  other  than  that  given  by  equation 
(2-48)  ,  y-,  =  x-j^,  the  command  input  vector  r_  must  be  written 
in  a  form  to  correspond  with  the  desired  values  of  the  state 
variables.   For  example  if  the  output  of  the  second-order  plant 
shown  in  Fig.  2-9  is  given  by 

yx  =  x-l  +  x2, 
and  the  desired  output  is  a  ramp, 

r(t)  =  gt 
then  for  zero  error 

y   =  gt  =  x   +  x2  (2-66) 

Taking  the  derivative, 

g  =  x^  +  x2 
or 

g  =  x2  +  0  ;  X]_  =  x  (2-67) 

for  equilibrium.   Solving  equations  (2-66)  and  (2-67)  simultan- 
eously, 
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xx  =  g(t-l)  =  r 

x2  =  g  =  r2  (2-68) 

Equation  (2-68)  gives  the  desired  values  of  the  state  variables 
and,  hence,  the  command  input  vector. 
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3.   GENERALIZED  PERFORMANCE  INDICES  AND  DYNAMIC  PROGRAMMING 

3.1  General 

In  this  chapter  dynamic  programming  will  be  applied  to 
optimize  the  performance  of  a  dynamic  system.   The  results 
are  applicable  to  both  sampling  and  continuous  systems,  a 
fact  which  will  be  discussed  later  in  the  chapter.   This 
approach  leads  to  a  stationary  control  law  for  the  infinite 
stage  regulator  problem. 

3.2  Problem  Definition  and  Development 

Consider  the  system  discussed  in  Chapter  1  described 
by  the  difference  equation 

x(k+l)  =0X(T)  x(k)  +  Dx(T)u(k)  (3-1) 

A  performance  criterion  is  established  to  measure  "how 
well"  the  system's  output  follows  the  command  input  r(t). 
This  is  called  the  performance  index  and  is  denoted  by  J. 
Fig.  3-1  shows  schematically  how  the  performance  index  is 
evaluated. 

Inputs:   The  method  of  dynamic  programming  requires  the 
knowledge  of  all  the  desired  states;  therefore,  the  system 
will  be  optimized  for  an  expected  class  of  command  inputs 


{ 


r(t)>  .   A  member  r(t)  of  this  class  of  inputs  is  defined  by 
r(t)  =  r^t)  ;  t»0 

=  0      ;  t*=  0  (3-2) 
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where 


r(t)  =0  (t)w(O)  = 


r1(t) 


r  (t) 
P 


(3-3) 


(7)  (t)  is  the  transition  matrix  of  a  pth  order  linear  differential 
equation  with  constant  coefficients  and  w(0)  is  an  arbitrary 
vector.  W     determines  the  class  of  inputs  for  which  the  system 
is  to  be  optimized  and  w(0)  determines  a  particular  member  of 
that  class. 
Example  1;   To  illustrate  the  formulation  of  (p   ,  suppose  a 


member  of  the  class  of  inputs  <r(t)>  satisfies  the  differential 


equation 


*2£(t) 
dt2 


=  0 


(3-4) 


The  equations  for  the  state  variables,  (2-49) ,  lead  to  the 
flowgraph  shown  in  Fig.  3-2.  (h     may  be  determined  by  inspec- 
tion of  the  flowgraph: 


0r(t)  =£ 


-i 


l/s   l/s 
0     l/s  J 


1  t 
0  1 


(3-5) 


Then,  from  equation  (3-3) , 

r  (t)  =  wx(0)  +  tw2(0) 

Example  2:   The  solution  to  the  harmonic  equation 

2 
wl  +U   w2  =  °  '    t-° 

yields  a  sinusoidal  signal.   This  equation  may  be  decomposed 
into 


(3-6) 


(3-7) 
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,  w2(0) 


Vs 


/s 


wt(0) 
i 


V/s 


'A 


W, 


w, 


w, 


Fig.  3-2       FLOWGRAPH     FOR    POLYNOMIAL 

EXAMPLE 


/s 


-u> 


w,(0) 


Fig.  3-3  FLOWGRAPH     FOR    SINUSOIDAL 

EXAMPLE 
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w      =   -L/w-l 


(3-8) 


From   the    state   equations   or    from   the    flowgraph   shown    in   Fig. 


3-3,  0     is    seen   to   be 


0r<t>    =*"' 


u 


2      ,   ,2 
s      +U 


-u 


2      .    .2 
s     +U 


2     x/    I2 

s      +L/ 


2      /   |2 


Then 


cosL7t        sinCJt 
-sinLJt        cosC7t 


r1(t)  =  w  (O)cosCJt  +  w2(0)sinLJt 


(3-9) 


(3-10) 


If  a  system  has  been  designed  for  optimal  performance  for 
a  particular  class  of  inputs,  it  will  approach  nearly  optimal 
performance  with  any  other  input  signal  which  can  be  closely 
approximated  by  an  r(t)  over  one  sampling  interval 
Performance  Index;   The  error  between  command  input  and  plant 
output  is  given  by 

(3-11) 

The  undesirability  of  error  is  not  usually  a  function  of  the 


A 


e(t)  =  r(t)  -  Yl(t) 


sign,  so  a  more  plausible  measure  would  be 

A 


or 


e(t)  =  |r(t)  -  Yi(t)\ 


e(t)  =  (f(t)  -  Yl(t)) 


(3-12) 
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The  second  is  commonly  used  because  of  its  mathematical 
convenience.   A  generalized  measure  of  error  is  defined  as  the 
quadratic  form 

zT(t)Qz_(t)>  0  (3-13) 

where  Q  is  an  (n+p)  by  (n+p)  positive  semi-definite  matrix, 
and  _z(t)  is  an  (n+p)  by  1  vector  defined  by 


z(t)  = 


x(t) 
r(t) 


(3-14) 


Example:   To  illustrate  the  preceding  development,  suppose 
the  desired  measure  of  error  is 

(r^t)  -  xx(t))2 
Then,  the  Q  matrix  is  given  by 

ql,l  =  qn+l,n+l  =  1 


ql,n+l  =  qn+l,l  =  _1 


all  other  elements  =  0 


(3-15) 


To  complete  the  notation  in  terms  of  the  combined  state  vector 
z_(t)  ,  define 

D(T)  = 

(3-16) 


0(T)  = 


D  (T) 
■-  r 


0X(T) 
0 


0 


0r(T) 


(3-17) 
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where  D  (T) , 0  (T)  andrf)  (T)  are  as  previously  defined,  and 
D  (T)  is  0_.  The  latter  follows  from  the  definition  of  r(t) 
in  equation  (3-3)  where  there  is  no  D  (T)  term. 

The  two  state  transition  difference  equations  are  com- 
bined as 

z(k+l)  =  0(T)z(k)  +  D(T)u(k)  (3-18) 

In  many  systems  the  control  energy  represents  cost  and, 
therefore,  it  may  be  desired  to  consider  control  energy  in  the 
measure  of  performance  as  well  as  the  system's  errors.   The 
quadratic  form  is  also  used  for  measuring  control  effort 
uT(t)Ru(t) 

As  stated  previously,  if  the  performance  of  a  system  is 
to  be  optimized,  a  criterion  must  be  selected  to  tell  "how 
well"  it  is  operating.   This  criterion  is  the  performance 
index,  which,  over  the  period  0*  t^  NT  is  given  by 

JN[z  (°)  '2i(T)  ,  .  •  •  ,u(  (N-l)  T)] 

NT 
=  f    jzT(t)Qz(t)  +  u*T(t)Ru*(t)A  (t)dt]         (3-19) 

0 

where  Q  and  R  are  proportional  weighting  matrices  which  com- 
promise between  minimization  of  errors  and  control  energy, 
/^(t)  is  a  positive  time-variant  weighting  function.   Note  that 
the  smaller  JN  the  more  satisfactory  the  system's  performance. 

The  foregoing  definition  is  very  general  and  by  varying 
Q,R  and  A(t)  the  index  is  adjusted  to  fit  problems  of  physical 
significance. 
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Example  1:   The  response  of  a  plant  is  never  instantaneous, 
and  as  a  result  for  some  period  of  time  following  the  applica- 
tion of  the  control  signal,  errors  are  inevitable.   As  time 
progresses,  these  errors  are  expected  to  become  smaller. 
Therefore,  it  may  be  desirable  to  have  A(t)  increase  with 
time.   If  the  errors  and  the  control  effort  are  to  be  measured 
only  at  the  sampling  instants  and  given  zero  weight  at  all 
other  times,  then, 

JN  =t   [A<k)  (zT(*)Qz(k)  +  uT(k-l)Ru(k-l))]      (3-20) 
k=l 

where  ^(k)  is  defined  as  the  value  of  A(t)  at  the  kth  sampl- 
ing instant.   if  the  weighting  of  errors  and  control  effort 
remains  the  same  at  all  sampling  instants,  then/.(k)  is 
dropped  from  the  performance  index. 

Example  2 ;   A  performance  index  concerned  only  with  the  error 
at  the  Nth  sampling  instant  is 

J   =  zT(N)Qz(N)  (3-21) 

N    —      ~ 

If  Q  is  the  identity  matrix,  this  index  is  the  Euclidean 
norm  of  the  system's  state  variables  at  the  end  of  the  Nth, 
or  final  stage  of  the  process.   Note  that  it  reflects  no 
concern  whatever  for  what  goes  on  in  the  early  stages  of  the 
process,  nor  for  the  control  effort  required.   It  reflects 
the  desire  to  bring  the  system  as  close  as  possible  to  the 
origin. 
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If  a  system  has  deadbeat  response  as  described  in  Chapter 
2,  it  is  possible  to  select  a  control  signal  that  will  make 
JN  =  0  for  a  finite  N  and  any  initial  state  z^(O)  . 
3.3  Recurrence  Relations  I 7j 

It  is  now  possible  to  state  the  problem  more  precisely. 
Given  a  linear  plant  defined  by  equation  (3-1)  and  a  class  of 
input  signals  defined  by  equations  (3-2)  and  (3-3) ,  find  the 
sampled  control  signal  u(k)  for  the  t^rO  which  minimizes  the 
performance  index.   That  is,  find  the  sequence  of  vectors 
u(0) ,  u(l) ,  u(2) , . . .  such  that  for  any  initial  state  z (0) , 
the  performance  index  J  ,  equation  (3-19) ,  is  minimized.   The 
performance  index  considered  is 

jfz(0)l  =£  fzT(k)Qz(k)  +  u  (k-l)Ru(k-l)        (3-22) 
*■    J   k=lL 

which  is  the  case  where  \$X)    is  dropped  from  equation  (3-20) . 

The  optimum  return  function  J°[_z(0)[  is  defined  as  the 

minimum  value  of  J  [z(0)j  and  has  the  known  simple  form 

J°[z(0)j  =  zT(0)PNz(0)  (3-23) 

which  is  proved  by  induction  later.   P   is  a  symmetric, 

N 

positive  semi-definite  matrix.   By  definition 

min  min 
JN+1|"z(0)J  =  u(0)  ,u(l),  .. 

.   N+l 
..u(N)]f=1[z T(k)Qz(k)  +  u^k-DRufk-Uj    (3-24) 
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z_(l)  is  determined  solely  by  u(0)  ,  which  enables  equation 

(3-24)  to  be  written  as 

min  <  min 

JN+l[-(0)]  =  u(0)^zT(l)Qz(D  +  uT(0)Ru(0)  +  u(l), 

min      min  n+1 

u(2)(...,u(N)  £   [z  (k)Qz(k)  +  uT(k-l)Ru(k-l) 
k=2  L 

(3-25) 

The  last  group  of  terms  is  exactly  J°[.z(l)   which  gives 

JN+l[z(0)]  =  £(0)   zT(l)Qz(l)  +  uT(0)Ru(0)  +  jSfi'd) 

(3-26) 

This  equation  can  be  arrived  at  directly  using  the 
principle  of  optimality,  which  states  that  the  minimum  cost 
of  an  N+1  stage  process  is  the  minimum  of  the  sum  of  the  cost 
of  the  first  stage  and  the  minimum  cost  of  the  N  remaining 
stages.   Note  that  the  arguments  of  the  state  and  control 

variables  increase  with  time,  whereas  the  subscript  on  J 

*  N 

decreases  with  time. 

Substituting  equations  (3-18)  and  (3-23)  into  (3-26) 

JN+lf*(0)]  " 


.'■<<.  J..1J   I  _ 

=  u(0)J[<^z(0)  +  Du(0)j  T(Q  +  PN)  [0z(O)  +  Du(0)j  +  u  (O)Ru(O) 

(3-27) 


min  f 

i 

Completing  the  square  on  the  right  side  of  (3-27)  and 
defining 

u'  (0)  =  Aivt_u1z(0)  (3-28) 

AN+1  =  -[DT(Q  +  PN)D  +  RJ_1DT(Q  +  PN)<7)  (3-29) 
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Equation    (3-27)    becomes 

min  t  T  T 

JN+l[-(0)]   =   u(0)i[u(0)     "  a"  (0)]   \>    (Q   +   PN)D   +    R][u(0)     -    u'  (0)] 

+    zT(0)(7)T(q   +    Pn)0z(O) 

"  ^T(0)Vl[DT(Q   +    PN)D   +   R]Vl   ^(0)  (3"30) 

The  control  u(0)  occurs  only  in  the  first  term.   If  the  matrix 

|D  (Q  +  P-JD  +  R  is  positive  definite  the  optimal  control  is 

unique.   Then,  the  minimum  value  of  the  first  term  is  zero 

and  occurs  when 

u(0)  =  u' (0)  (3-31) 

The  matrix  will  be  positive  definite  if  R  is  positive  definite, 

or  if  Q  is  positive  definite  and  the  columns  of  D  are  linearly 

independent.   It  will  not  be  positive  definite  if  R  =  0,  and 

the  columns  of  D  are  linearly  dependent.   In  other  cases  the 

matrix  might  be  singular. 

Equations  (3-31),  (3-28)  and  (3-29)  define  the  optimal  value 

of  u(0)  . 

The  recurrence  relation  for  P     is  determined  by  equating 

N+l 

(3-30)     and    (3-23)    when   u(0)    =   u'  (0)  , 

jn+i[-(0)]    =  -*T(°)0T(Q  +  pN>$-(0) 

-    zT(0)ATn+1[dT(Q   +    PN)D   +    RJAN+1z(0) 

=    zT(0)PN+i^(0)  (3-32) 
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Since  equation  (3-32)  must  hold  for  all  jz  (0) ,  the  recurrence 
relation  becomes 

PN+1  =0T(Q  +  PN}  (0+  DAN+1>  (3"33> 

where 

-  aTn+i[dT(Q+  Vd  +  R]  =0T(Q  +  VD  (3_34) 

or 

AN+1  =  "[dT(Q  +  PN)D  +  r]"1dT(Q  +  PN)0         (3-34a) 
has  been  used  to  simplify  (3-33) . 

Equation  (3-32)  shows  that  if  the  quadratic  form  for  J° 
is  correct,  then  J  ■J?-,    has  the  same  form.   The  quadratic  form 
is  trivially  correct  for  J0  since 

Jq[^(0)]  =  °  (3-35) 

for  all  z^O)  .   To  complete  the  mathematical  induction,  the  form 

for  J°  must  be  shown  to  be  correct.   J?  is  determined  from 

1  1 

equation  (3-32)  and  (3-29)  noting  that  Pq  =  0.   Equation  (3-34) 
is  again  used  to  simplify  the  result. 

Jlfz(0)]  =  zT(O)0TQ  [0  -  D(DTQD  +  R)_1DQ0]z(O) 

=  zT(0)P1z(0)  (3-36) 

J?[z_(0)l  has  the  desired  quadratic  form. 
3.4  Solution  of  the  Problem 

The  solution  proceeds  using  the  recurrence  relations, 
equations  (3-33)  and  (3-34a).   Since  J  fz(0)]  =0,  P   =  0; 
calculate  A^ .   From  A^  and  P~  calculate  P,.   Continue  this 
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process  until  all  A  of  interest  are  calculated.   If  the 

N 

plant  is  controllable,  A  will  tend  to  a  limit  as  N  increases. 

N 

Therefore,  for  the  infinite  stage  regulator  problem  the 
optimal  control  takes  the  form  of  a  stationary  linear  function 
of  the  states,  which  is  independent  of  the  sampling  instants  kT: 

u(k)  =  ANz(k)  (3-37) 

However,  if  it  is  desired  to  optimize  J„  over  a  finite  number 
of  samples,  then  the  optimal  control  takes  the  form  of  a 
different  linear  function  of  the  states  at  each  sampling 
instant. 

u(k)  =  A(k)z(k)  (3-38) 

An  advantage  of  this  design  procedure  is  that  it  always 
results  in  a  stable  control  system.   This  is  due  to  the  state 
method  of  describing  the  system's  dynamics,  and  to  the  use  of 
dynamic  programming  to  obtain  the  solution   3  . 

The  resulting  system  is  shown  in  Fig.  3-4.   Note  that 
it  is  a  multiloop  feedback  system,  requiring  measurement  of 
the  plant's  states  as  well  as  the  input  states. 

If  T  becomes  small,  relative  to  the  plant's  time  constants, 
the  sampled  signal  u*(t)  is  nearly  the  same  as  the  continuous 
signal  u(t) .   Then  the  sample  and  hold  element  can  be  removed 
from  the  system,  resulting  in  a  continuous  system.   If  R  =  0, 
and  if  T  approaches  zero,  then  the  control  input  becomes 
unbounded.   For  this  reason  R  must  be  greater  than  zero  if  T 
is  going  to  be  very  small,  or  if  a  continuous  system  is  desired. 
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A  comment  on  notation  is  made  here.   When  the  control 

input  u  is  a  vector  quantity  the  feedback  control  matrix  will 

be  denoted  as  A„,  whereas,  if  the  control  input  is  a  scalar 

quantity,  the  feedback  control  matrix  will  be  denoted  as 
T 
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4.   NUMERICAL  EXAMPLES 

To  illustrate  the  control  laws  derived  in  Chapters  2 
and  3,  the  following  examples  are  presented.   All  simula- 
tions were  done  with  the  CDC  1604  digital  computer. 
4.1  Deadbeat  Response 

The  third  order  system  shown  in  Fig.  4-1  was  simulated. 
The  descriptive  equations  are 


x(k+l)  =0x(k)  +  du(k) 


u(k)  =  a^(k) 


The  state  transition  matrix®,  and  the  d  matrix  for  a  sampl- 
ing period  T  =  77/4  are 

0  = 


1  .7091  .2509 
0  .7265  .5585 
0  -.6088   .3914 


d  =  f. 06965 


2509 


,5585 
Using  the  deadbeat  performance  index 

JN  =  xT(N)  I  x(N) 
and  the  recurrence  relations  derived  in  Chapter  3,  the 
following  control  law  was  computed,  where,  due  to  the  use  of 
a  scalar  control,  N  is  equal  to  the  order  of  the  plant. 
u(k)  =  %2i(k)  ~  |ai  a2  a3J  2£(k) 

=  [-2.741672  -3.127373  -2.045130J  x(k) 
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The  same  gain  matrix  was  obtained  using  the  methods  of 
Chapter  2.   The  system's  trajectory  is  presented  in  Table  4-1 


■  -  ■■  1 
Stage (k) 

.  _ 

Control  u(k) 

x1(k) 

x2(k) 

x3(k) 

Initial  (0) 

-2.7416 

1.000000 

0 

0 

(1) 

3.064975 

.808983 

-.687893 

-1.531268 

(2) 

-1.711838 

.150569 

-.585992 

1.531268 

Final    (3) 

0 

0 

0 

0 

TABLE  4-1  SYSTEM  TRAJECTORY 
FOR  DEAD BE AT  EXAMPLE 

The  recurrence  relations  also  yield  a  series  of  gain 
matrices  forming  a  control  law  which  is  a  different  function 
of  the  states  at  each  sampling  instant, 

u(k)  =  aT(k)x(k) 
This  control  law  gives  the  identical  control  sequence  u(0), 
u (1)  ,  . . . , u (N-l)  as  the  stationary  control  law  above. 
4.2  Infinite  Stage  Regulator 

The  optimum  control  law  for  the  infinite  stage  regulator 
is  also  in  the  form  of  a  stationary  linear  function  of  the 
state  variables.   The  control  gains  and  the  cost  function  can 
be  computed  by  the  recurrence  relations  of  Chapter  3.   After 
the  optimum  gains  are  established,  the  system's  trajectory  is 
computed  from  the  difference  equation  of  the  system. 

This  example  utilizes  the  same  third  order  plant  as  the 
deadbeat  example,  Fig.  4-1.   The  problem  was  formulated  as 
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in  Chapter  3  and  the  resulting  system  is  as  shown  in  Fig.  3-4 
The  performance  index  is 
N 
k=l 
where 


JN  =£    [zT(k)Qz(k)    +   u2(k-l)R] 


Q   =         100-10  R   =    1.0 

0      10      0-1  T   =  Tt/A 

0      0      10      0 

-10      0      10 

0-1001 

For  a  polynomial  input  such  as  described  in  equations  (3-4) 

through  (3-6) ,  the  gain  matrix  was  computed  to  be 

aT  =  [-.581355  -.626587  -.922714  .581355  1.7165871 
"~N    L  J 

The  command  inputs  used  were  a  unit  step  and  a  unit  ramp, 

and  the  results  are  shown  in  Fig.  4-2  and  4-3  respectively. 

Both  step  and  ramp  responses  were  within  1%   of  the  desired 

values  at  the  end  of  eleven  samples.   Note  that  the  gain 

matrix  has  the  pattern  expected  for  a  system  with  one  free 

integrator  as  discussed  in  Chapter  2;  i.e.,  that  a   ,  =  -a, 

n+1      1 

and  a     =  -(a„  +  d)  =  -(a„  -  1.09).   In  the  development  of 
Chapter  2,  it  was  shown  that  such  a  system  would  reach  equilib- 
rium with  step  and  ramp  command  inputs  only  if  in  addition  to 
the  optimum  plant  state  feedback  gains,  optimum  gains  on  the 
command  input  states  were  incorporated;  i.e.,  if  the  optimum 
gains  are  not  applied  to  the  input  states,  the  system's  ramp 
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response  will  have  a  steady  state  error.   A  ramp  command 
input  was  applied  to  the  example's  system,  without  the  gains 
on  the  input  states,  and  the  response  is  shown  in  Fig.  4-4. 

For  a  sinusoidal  command  input  as  developed  in  equations 
(3-7)  through  (3-10)  ,  the  gain  matrix  was  computed  to  be 

a^  =  [-.581355  -.626587  -.922714  .37783  .581434] 
Note  that  the  first  n  elements  are  the  same  as  for  the  poly- 
nomial command  input,  but  a   n  and  a   ~  are  different.   The 

r  n+1      n+2 

system's  trajectory  is  shown  in  Fig.  4-5. 

Fig.  4-6  shows  the  trajectory  for  the  same  system  with 
a  sinusoidal  input  but  with  the  weighting  matrix  Q  as  given 
by  equation  (3-15) .   The  gain  matrix  was  computed  to  be 

aT  =  [-4.484  -4,275  -2.313  4.080  1.8151 
-N    L  J 

4.3  The  Finite  Regulator  Problem  (CASE  STUDY) 

As  stated  in  Chapter  3,  if  it  is  desired  to  optimize 
a  system's  performance  over  a  finite  number  of  samples  M, 
where  M  is  either  smaller  or  not  much  greater  than  the  number 
of  recurrence  steps  required  for  A  to  tend  to  a  limit,  the 
resulting  control  takes  the  form  of  a  different  linear  func- 
tion of  the  states  at  each  sampling  instant, 

u(k)  =  A(k)x(k) 
A  problem  of  this  type  would  occur  if  the  command  input  to  the 
system  were  reinitialized  every  M  samples.   An  example  might 
be  a  missile  that  is  using  a  search-while-track  radar  and  is 
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receiving  new  information  at  the  rate  MT,  where  T  is  the 
system's  sampling  period. 

To  physically  implement  such  a  system  would  require  a 
computer  to  reset  the  control  feedback  gains  at  each  sampling 
instant.  The  engineering  complexity  involved  with  such  a 
system  warrants  investigating  a  system  with  fixed  control 
feedback  gains.  Though  not  optimal,  this  system  might  be 
considered  superior  due  to  other  criteria  such  as  weight, 
cost,  etc.  This  form  of  control  will  be  referred  to  as  sub- 
optimal  in  this  section. 

An  investigation  of  suboptimal  control  was  accomplished 
by  simulating  second,  third  and  fourth  order  systems  and 
comparing  the  performance  to  optimal.   The  performance  index 

used  was 

N    T  2       i 

JN  =  £   [£  (k)Qx(k)  +  u  (k-l)Rj 


and  most  of  the  investigation  was  made  with  the  second  order 
system  of  Fig.  4-7 . 

The  state  transition  matrix  (p,    and  the  d  matrix  for  a 
sampling  period  of  0.1  are 
Cb  =  \   .9952    .0931 
_-.0931    .8636 
d  =T  .004768 
.0931 
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The  weighting  matrices  are 
Q  =  fl    0 
0    1 


R 


=  [i-o] 

The  system  was  run  for  five  samples. 

It  was  observed  that  if  the  system  was  run  for  five  samples 
using  A^  as  the  fixed  control  feedback  matrix,  the  performance 
was  not  greatly  degraded  from  that  of  optimal.   This  was  also 
true  of  the  higher  order  systems.   The  amount  of  performance 
degradation  that  is  acceptable  is,  of  course,  an  engineering 
decision  based  on  the  physical  aspects  of  the  problem. 

Runs  were  also  made  using  the  values  that  A   tended  to 
in  the  limit „   Even  though  these  values  gave  optimal  perform- 
ance in  the  infinite  stage  regulator  problem,  the  total  cost 
was  higher  in  the  five-stage  finite  regulator  problem  than 
that  obtained  using  the  gain  matrix  Ar. 

The  elements  of  Ac  were  then  perturbed  to  see  if  the 
performance  could  be  improved.   This  led  to  an  exhaustive 
search  which  resulted  in  a  gain  matrix  which  differed  signifi- 
cantly from  Ar*  and  the  resulting  performance  was  near  optimal. 
During  the  search  the  system's  parameters  and  the  initial 
conditions  were  held  constant. 

Using  the  gains  obtained  in  the  search,  various  initial 
conditions  were  tried,  generally  resulting  in  performance 
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close  to  that  obtained  with  A5.   The  results  of  this  case 

study  are  presented  in  Table  4-2.   The  column  A  contains 

P 

the  costs  using  gains  obtained  by  perturbation. 


Initial 
Conditions 

5   T 
Total  Cost  J5  =£  1  x  (k)lx(k)  + 

u2(k-l)j 

x(0) 

Optimal 

Using 
A5 

Using 
Aoo 

Using 

A 
P 

y 

i 

48.658 

48.712 

55.159 

48.7714 

i" 

i 

* 

8.0210 

8.0500 

9.5956 

8.0214 

'i 

3 

28.0741 

28.2912 

34.2085 

28.1898 

10" 

.2 

488.0655 

488.2655 

531.8718 

490.7636 

.2 
10 

230.8611 

233.1031 

270.0404 

233.5962 

TABLE  4-2   RESULTS  OF  CASE  STUDY  OF  SUBOPTIMAL 
CONTROL  FOR  THE  FINITE  STAGE  REGULATOR  PROBLEM 

*  Initial  conditions  used  during  exhaustive  search 


Gain  Matrices: 

A5  =  [-.038279   -.26493l] 
A   =  [-.376449   -.509776J 


-[ 


045198   -.302418 


] 


Optimal  (in  order  of  application) : 


A5  =  [-.0 

A4  -  b° 


38279   -.264931 


21133   -.243223 
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A3  =  [-.007999   -,,216953 
A2  =  [  .000587   -.183544] 
A1  =  ("  .004439   -.139468] 
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5 .       CONCLUS IONS 
5.1   Summary 

5.1.1  Deadbeat  Response 

The  optimum  control  law  derived  for  deadbeat  response 
takes  the  form  of  a  stationary  linear  function  of  the  state 
variables,  independent  of  the  sampling  instant  kT: 

u(k)  =  A^C(k) 
where  N  is  the  number  of  stages  required  to  bring  the  system 
to  the  origin,  N<  n. 

It  has  also  been  shown  that  by  using  dynamic  program- 
ming to  minimize  the  performance  index 

J  =  xT(N)Qx(N) 
where  x(N)  is  the  final  stage  vector  and  Q  is  the  identity 
matrix,  the  resulting  control  sequence  will  give  deadbeat 
performance.   The  control  law  may  be  either  stationary  or  a 
different  linear  function  of  the  states  at  each  sampling 
instant  kT.   The  control  sequence  is  the  same  in  both  cases. 

u(k)  =  AN*(k)  =  A(k)x(k) 

5.1.2  The  Infinite  Stage  Regulator 

The  optimal  control  law  derived  for  the  infinite 
stage  regulator  is  in  the  form  of  a  stationary  linear  func- 
tion of  the  state  variables.   The  procedure  for  obtaining  the 
constant  control  gains  is  well  established  I  3,  7J  ,  and  is  given 
in  Chapter  3. 
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5.1.3  The  Finite  Stage  Regulator 

The  optimum  control  law  derived  for  the  finite 
stage  regulator  is  a  different  linear  function  of  the  state 
variables  at  each  sampling  instant, 

u(k)  =  A(k)x(k) 

If  it  is  desired  to  achieve  the  best  performance 
possible  with  a  stationary  control  law,  the  results  of  a 
case  study  tend  to  indicate  that  using  A^  as  the  fixed  gain 
matrix,  where  M  is  the  number  of  sample  periods,  results  in 
a  system's  performance  not  greatly  degraded  from  optimum. 
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APPENDIX  I 
Derivation  of  Difference  Equations  Representing 
a  Sampled-Data  Dynamic  System 
A  block  diagram  of  the  system  discussed  in  this  Appendix 
is  shown  in  Fig.  1-1. 

Plant:   A  linear,  time  invariant  dynamic  system  with  state 
equations,  which  in  vector  notation  may  be  written 

x(t)  =  Ax(t)  +  Bu(t)  (1-1) 

£(t)  =  Cx(t)  +  Du(t)  (1-2) 

The  Laplace  transformation  of  equation  (1-1)  is 

sX(s)  =  x(O-)  =  AX(s)  +  BU(s)  (1-3) 

Rearranging, 

sX(s)  -  AX(s)  =  x(O-)  +  BU(s) 
(si  -  A)X(s)  =  x(O-)  +  BU(s) 
X(s)  =  (si  -  A)x(O-)  +  (si  -  A)"  BU(s)  (1-4) 


Let 


</>(s)  i  (si  -A)"1 

g6(t)  ix_1[0(S)] 


The  solution  of  (1-1)  becomes 

x(t)  =CJ)(t)x(0~)    +   f(f>{t-T)  Bu(T)  dT  (1-5) 

0 

where  it  is  assumed  that  u(t)  =  0  ;  t<  0,  and(7)(t)  is  causal. 
More  generally:   (initially  t  =  t^  vice  zero) 
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x(t)  =0(t-tQ)x(to)  +r<0(t-r)Bu(T)dT 


(1-6) 


^s 


natural  response   forced  response 
For  an  interval  of  time  T  set: 


t   =  kT 


t  =  (k+l)T 
Then,  (1-6)  becomes: 


(k+l)T 


x((k+l)T)  =  0(T)x(kT)  +f  0  ((k+l)T-T)Bu(T)dT 


kT 


where  u(t)  is  piecewise  constant,  i.e., 


ux(t)  =  u  (kT)  ^ 


(1-7) 


u2(t)  =  u2(kT) 


Um(t)  =  um(kT) 


)    for  kT*  t  -c(k+l)T 


Then, 


(k+l)T 
kT 


T 


f   (73((k+DT-r)Bu(r)dr  =1  faer-r)  mtU 


(kT) 


=  D(T)u(kT) 
Then,  (1-7)  becomes 

x(k+l)T  =0(T)x(kT)  +  D(T)u(kT) 
Some  Properties  of  the  State  Transition  Matrix 

1)  0(0)  is  an  identity  matrix 
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2)  (7>(kT)     =     ((}/)(T)) 

.jl,,      .  ATk  .    AT\k 

Since   by  definition,  (p(kT)    =   e  (e      ) 

3)  0(-kT)    ps   (0(kT))_1 

,jl  -AkT  AkTv-1 

SinceCp(-kT)    =   e  =    (e         ) 
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